In computing, optimization is the process of modifying a system to make some aspect of it work more efficiently or use fewer resources. For instance, a programmer may optimize a computer program so that the program executes more rapidly, is capable of operating with less memory or other resources, or draws less power. The system may be a single computer program, a collection of computers, or even an entire network such as the Internet. Although the word “optimization” shares the same root as “optimal,” it is rare for the process of optimization to produce a truly optimal system. The optimized system will typically only be optimal in one application or for one audience. One might reduce the amount of time that a program takes to perform some task at the price of making it consume more memory. In an application where memory space is at a premium, one might deliberately choose a slower algorithm in order to use less memory. Often there is no “one size fits all” design which works well in all cases, so engineers make trade-offs to optimize the attributes of greatest interest. Thus, a programmer may halt the process of optimization before a completely optimal solution has been reached.
Optimization can be automated or performed by programmers. Programmers usually undertake optimizing a whole system because it is too complex for automated optimizers. In this situation, programmers or system administrators explicitly change code so that the overall system performs better. Although manual optimization can produce better efficiency, it is far more expensive than automated optimizations. Programmers sometimes believe they have a clear idea of where the bottleneck is, but intuition is frequently wrong. Optimizing an unimportant piece of code will typically do little to help the overall performance. Programmers use applications called profilers (or performance analyzers) to find the sections of the program that are taking the most resources—the bottleneck. When the bottleneck is localized, optimization usually starts with a rethinking of the algorithm used in the program: more often than not, a particular algorithm can be specifically tailored to a particular problem, yielding better performance than a generic algorithm. In many cases, it is difficult or impossible for a human to optimize a particular system, because he/she may not know what resources will be available in the environment in which the program will actually run. For example, an Internet application developer does not know how much bandwidth will be available to each machine running the application, and in fact the availability of that particular resource is constantly changing. As another example, take an application with parts that can run in parallel, where the programmer wants to decide how many parts to run in parallel. He cannot optimize this statically, because he does not know at design time what machines the application will run on.
Algorithms that optimize the outputs of dynamic systems typically measure the effect of changes in the inputs that are under the algorithm's control (“control variables”) on the output. The system may have many other inputs which are outside of the algorithm's control, but that have as much or more effect on the output of the system than the control variables. The value of each of these external inputs may not be known, and even if they were known, a programmer may not know how they would be reflected in the output the programmer is trying to optimize. Furthermore, the ability of the algorithm to obtain precise measurements of the system's output is often limited. This results in a “noisy” view of the output data, making it even more difficult to measure the effect of an input change.
For example, in a dynamic thread pool, a control variable might be the number of threads in the pool (“thread count”), and the output to optimize might be the rate at which work completes on those threads (“throughput”). A change in the number of threads will have an effect on the throughput, but so will a change in the workload being executed in the pool, a change in resource demands of other processes running on the machine, and so forth. Because these external variables are outside of our control, we cannot hold them constant while we measure the effect of a throughput change.
Another example would be optimizing throughput on a network by finding an optimal packet size. One would like to know how a change in packet size affects the network's throughput, but would have to isolate this from other changes due to external factors such as changes in user access patterns, variations in incoming external traffic, failure of load-balanced redundant systems, and so on. Noise may be introduced into any throughput measurement for similar reasons to the thread pool example.